[WEB-4956] Add no_tools_found Sentry event to diagnose empty discovery scans#215
Merged
Merged
Conversation
A scan that finds zero tools was previously silent — indistinguishable from an errored scan, a missed detection, or a genuinely empty device. Emit one fire-and-forget Sentry warning (phase=no_tools_found) carrying the discriminators needed to tell them apart: - homes_enumerated: pre-fallback enumerated-user count (0 = enumeration miss; the existing post-fallback user_count masks 0 as 1) - users_scanned, used_fallback_user, is_root, os, duration_ms Counts/booleans only (no PII); reuses the existing curl-based report_to_sentry; wrapped so telemetry can never raise into or slow the scan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vigneshsubbiah16
left a comment
Contributor
There was a problem hiding this comment.
🛡️ Automated Security Review (consensus)
0 findings — 0 high-confidence, 0 to triage. Reviewers: Cursor, Claude, Semgrep, Gitleaks.
✅ Security consensus: no issues found. (reviewers: Cursor, Claude, Semgrep, Gitleaks)
🤖 consensus review · reviewers: Cursor,Claude,Semgrep,Gitleaks · head 29e59bce · 2026-06-26T03:30Z
MohamedAklamaash
approved these changes
Jun 26, 2026
A noisy empty scan (many earlier detector errors) could exhaust the shared 30-event per-run cap before the terminal no_tools_found summary runs, dropping exactly the diagnostic you most need on a likely missed-detection. Add a `priority` flag to report_to_sentry that bypasses the COUNT cap only — it still honors the circuit breaker (a dead transport can't be helped) and dedup (no spam) — and pass priority=True for the no_tools_found event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The priority flag bypassed the count cap but the circuit breaker — tripped by 3 consecutive, possibly transient, send failures — could still skip the terminal no_tools_found event without even an attempt. Allow a priority event ONE bounded attempt past the breaker (at most one ~4s curl at end of run) so a transient mid-scan outage doesn't drop the terminal diagnostic. Dedup is still honored, so it can never spam. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
🛡️ Automated Security Review (consensus)0 findings — 0 high-confidence, 0 to triage. Reviewers: Lead, Claude, Semgrep, Gitleaks. ✅ Security consensus: no issues found. (reviewers: Lead, Claude, Semgrep, Gitleaks) 🤖 consensus review · reviewers: Cursor,Claude,Semgrep,Gitleaks · head |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
phase=no_tools_found) on the no-tools path, carrying the discriminators needed to diagnose it at a glance.Why
homes_enumeratedis the key fieldThe existing metrics report
user_count = len(all_users)after theif not all_users: all_users = [get_user_info()]fallback, so a true enumeration miss (0) is masked as1. We capture the count before the fallback:homes_enumerated == 0⇒ enumeration miss (the macOS/AD network-account class);> 0⇒ genuinely empty / timing race.Changes
scripts/coding_discovery_tools/ai_tools_discovery.pyhomes_enumeratedbefore the user-enumeration fallback.if not tools:, emitreport_to_sentry(RuntimeError("Discovery found no tools"), level="warning")with context:phase,homes_enumerated,users_scanned,used_fallback_user,is_root(whenos.getuidexists),os,duration_ms(plusdevice_id/run_id/system_userfrom the existingsentry_ctx).tests/test_discovery_flow.pyTestNoToolsSentryEvent: fires exactly once on a zero-tool scan with the right fields; does not fire when tools are found.Diagnosing from Sentry
Filter
phase:no_tools_found, then read per event:homes_enumerated:0→ enumeration miss (network/AD account dropped)homes_enumerated:>0→ genuinely empty disk at scan time (timing race / true-empty)is_root:true→ ran in MDM/SYSTEM contextConstraints honored
try/except: pass; telemetry can never raise into or slow the scan.report_to_sentry(no new network code; no urllib).tools == []; relies onreport_to_sentry's existing per-run cap + dedup (uniquephase) + circuit breaker.Known limitation (deliberate, minimal scope)
Discriminator fields land in Sentry
extra, not searchable tags — onlyphase:no_tools_foundis filterable; values are readable per-event but not facetable/alertable without a follow-up_SENTRY_TAG_KEYSaddition. No backend/DeviceScan/migration changes (intentionally out of scope).Test plan
python3 -m pytest tests/test_discovery_flow.py -v— new tests passtests/suite green (69 passed), no regressions🤖 Generated with Claude Code
Note
Low Risk
Observability-only changes with fire-and-forget Sentry calls; scan completion path and backend payloads are unchanged except for optional warning telemetry.
Overview
Zero-tool discovery scans were silent in observability—hard to distinguish enumeration misses, timing races, and genuinely empty devices.
The scan now records
homes_enumeratedbefore the current-user fallback (so0is not masked as1in metrics). When no tools are found, it sends one fire-and-forget Sentry warning (phase=no_tools_found) with discriminators:homes_enumerated,users_scanned,used_fallback_user,is_root,duration_ms, plus existing run context. Telemetry is wrapped so it cannot fail the scan.report_to_sentrygains apriorityflag so this terminal diagnostic can bypass the per-run event cap and circuit breaker (still deduped). Tests cover zero-tool vs found-tool behavior and priority send semantics.Reviewed by Cursor Bugbot for commit ab61e8e. Bugbot is set up for automated code reviews on this repo. Configure here.
Greptile Summary
This PR adds Sentry visibility for discovery scans that find no tools. The main changes are:
no_tools_foundwarning with scan context when no tools are detected.Confidence Score: 5/5
This looks safe to merge.
Important Files Changed
report_to_sentryfor terminal diagnostics while preserving duplicate suppression.Reviews (3): Last reviewed commit: "Let priority Sentry events bypass the ci..." | Re-trigger Greptile